Why Not Run the Efficient Global Optimization Algorithm with Multiple Surrogates?

Surrogate-based optimization has become popular in the design of complex engineering systems. Each optimization cycle consists of analyzing a number of designs, fitting a surrogate, performing optimization based on the surrogate, and finally performing exact simulation at the design obtained by the optimization. Adaptive sampling algorithms that add one point per cycle are readily available in the literature. They use uncertainty estimators to guide the selection of the next sampling point(s). The addition of one point at a time may not be efficient when it is possible to run simulations in parallel. So we propose an algorithm for adding several points per optimization cycle based on the simultaneous use of multiple surrogates. The need for uncertainty estimates usually limits adaptive sampling algorithms to surrogates such as kriging and polynomial response surface because of the lack of uncertainty estimates in the implementation of other surrogates. We import uncertainty estimates from surrogates having such estimates to use with other surrogates such as support vector regression models. The approach was tested on two analytic examples for nine basic surrogates including kriging, radial basis neural networks, linear Shepard and support vector regression. For these examples we compare our approach with traditional sequential optimization based on kriging. We found that our approach was able to deliver better results in a fraction of the optimization cycles needed by the traditional kriging implementation.